Audio Response Playback

ABSTRACT

A computing device is configured to perform functions comprising: receiving via a network microphone device of a media playback system, a voice command detected by at least one microphone of the network microphone device, wherein the media playback system comprises a plurality of zones, and the network microphone device may be a member of a default playback zone. The computing device may be further configured to perform functions comprising: dynamically selecting an audio response zone from the plurality of zones to play an audio response to the voice input and foregoing selection of the default playback zone. The selected zone may comprise a playback device, and the dynamically selecting may comprise determining that the network microphone device is paired with the playback device. The computing device may cause the playback device of the selected zone to play the audio response.

CROSS-REFERENCE

This application claims the benefit of priority under 35 USC § 119(e) toU.S. Provisional Application Ser. No. 62/298,418 filed on Feb. 22, 2016and entitled “AUDIO RESPONSE PLAYBACK”, U.S. Provisional ApplicationSer. No. 62/312,350 filed on Mar. 23, 2016 and entitled “VOICE CONTROLOF A MEDIA PLAYBACK SYSTEM”, U.S. Provisional Application Ser. No.62/298,425 filed on Feb. 22, 2016 and entitled “MUSIC SERVICESELECTION”, U.S. Provisional Application Ser. No. 62/298,350 filed onFeb. 22, 2016 and entitled “METADATA EXCHANGE INVOLVING A NETWORKEDPLAYBACK SYSTEM AND A NETWORKED MICROPHONE SYSTEM”, U.S. ProvisionalApplication Ser. No. 62/298,388 filed on Feb. 22, 2016 and entitled“HANDLING OF LOSS OF PAIRING BETWEEN NETWORKED DEVICES”, U.S.Provisional Application Ser. No. 62/298,410 filed on Feb. 22, 2016 andentitled “DEFAULT PLAYBACK DEVICE(S)”, and claims priority to, and is acontinuation of U.S. patent application Ser. No. 15/237,133, filed onAug. 15, 2016, entitled “Audio Playback Response,” the contents each ofwhich are herein incorporated by reference in their entireties.

This application is also related to U.S. Provisional Application Ser.No. 62/298,439 filed on Feb. 22, 2016 and entitled “CONTENT MIXING”,U.S. Provisional Application Ser. No. 62/298,433 filed on Feb. 22, 2016and entitled “ROOM-CORRECTED VOICE DETECTION”, and U.S. ProvisionalApplication Ser. No. 62/298,393 filed on Feb. 22, 2016 and entitled“ACTION BASED ON USER ID”, the contents each of which are hereinincorporated by reference in their entireties.

FIELD OF THE DISCLOSURE

The disclosure is related to consumer goods and, more particularly, tomethods, systems, products, features, services, and other elementsdirected to media playback or some aspect thereof.

BACKGROUND

Options for accessing and listening to digital audio in an out-loudsetting were limited until in 2003, when SONOS, Inc. filed for one ofits first patent applications, entitled “Method for Synchronizing AudioPlayback between Multiple Networked Devices,” and began offering a mediaplayback system for sale in 2005. The Sonos Wireless HiFi System enablespeople to experience music from many sources via one or more networkedplayback devices. Through a software control application installed on asmartphone, tablet, or computer, one can play what he or she wants inany room that has a networked playback device. Additionally, using thecontroller, for example, different songs can be streamed to each roomwith a playback device, rooms can be grouped together for synchronousplayback, or the same song can be heard in all rooms synchronously.

Given the ever growing interest in digital media, there continues to bea need to develop consumer-accessible technologies to further enhancethe listening experience.

BRIEF DESCRIPTION OF THE DRAWINGS

Features, aspects, and advantages of the presently disclosed technologymay be better understood with regard to the following description,appended claims, and accompanying drawings where:

FIG. 1 shows an example media playback system configuration in whichcertain embodiments may be practiced;

FIG. 2 shows a functional block diagram of an example playback deviceaccording to aspects described herein;

FIG. 3 shows a functional block diagram of an example control deviceaccording to aspects described herein;

FIG. 4 shows an example controller interface according to aspectsdescribed herein;

FIG. 5 shows an example plurality of network devices according toaspects described herein;

FIG. 6 shows a function block diagram of an example network microphonedevice according to aspects described herein;

FIG. 7 shows a flow diagram of an example method of playing an audioresponse according to aspects described herein; and

FIG. 8 shows a flow diagram of blocks 708 and 712 of FIG. 7 in moredetail according to aspects described herein.

The drawings are for the purpose of illustrating example embodiments,but it is understood that the inventions are not limited to thearrangements and instrumentality shown in the drawings.

DETAILED DESCRIPTION I. Overview

Some embodiments described herein involve playing an audio response to avoice command in a media playback system and selecting a response deviceor zone to play the audio response to the voice command. In oneembodiment, the response device or zone may be predetermined, and inanother embodiment, the response device or zone may be dynamicallyselected based on context information.

As indicated above, the examples provided herein involve determining oneor more devices or zone(s) to play an audio response to a voice commandin a media playback system. In one aspect, a method is provided. Themethod may involve receiving a voice command from a microphone deviceassociated with a media playback system. The media playback system mayinclude one or more zones. At least one playback device of the one ormore playback devices or at least one zone may be selected to play theaudio response based on at least one of pairing information orcapabilities of the zone, and the selected at least one playback deviceor zone is caused to play the audio response.

In another aspect, a non-transitory computer-readable medium isprovided. The non-transitory computer readable medium has stored thereoninstructions executable by a computing device to cause the computingdevice to perform functions. The functions include receiving a voicecommand from a microphone device associated with a media playbacksystem, and the media playback system may include one or more zones,selecting at least one playback device of the one or more playbackdevices or a zone to play the audio response based on contextinformation, and causing the selected at least one playback device orzone to play the audio response.

In yet another aspect, an apparatus is provided. The apparatus includesa processor and a memory. The memory has stored thereon instructionsexecutable by the apparatus to cause the apparatus to perform functions.The functions include receiving a voice command via a microphone deviceassigned to a zone which includes one or more playback devices,selecting at least one playback device of the one or more playbackdevices to play the audio response based on at least one of pairinginformation or capabilities of the zone, and causing the selected atleast one playback device to play the audio response.

While some examples described herein may refer to functions performed bygiven actors such as “users” and/or other entities, it should beunderstood that this is for purposes of explanation only. The claimsshould not be interpreted to require action by any such example actorunless explicitly required by the language of the claims themselves. Itwill be understood by one of ordinary skill in the art that thisdisclosure includes numerous other embodiments.

II. Example Operating Environment

FIG. 1 shows an example configuration of a media playback system 100 inwhich one or more embodiments disclosed herein may be practiced orimplemented. The media playback system 100 as shown is associated withan example home environment having several rooms and spaces, such as forexample, a master bedroom, an office, a dining room, and a living room.As shown in the example of FIG. 1, the media playback system 100includes playback devices 102-124, control devices 126 and 128, and awired or wireless network router 130.

Further discussions relating to the different components of the examplemedia playback system 100 and how the different components may interactto provide a user with a media experience may be found in the followingsections. While discussions herein may generally refer to the examplemedia playback system 100, technologies described herein are not limitedto applications within, among other things, the home environment asshown in FIG. 1. For instance, the technologies described herein may beuseful in environments where multi-zone audio may be desired, such as,for example, a commercial setting like a restaurant, mall or airport, avehicle like a sports utility vehicle (SUV), bus or car, a ship or boat,an airplane, and so on.

a. Example Playback Devices

FIG. 2 shows a functional block diagram of an example playback device200 that may be configured to be one or more of the playback devices102-124 of the media playback system 100 of FIG. 1. The playback device200 may include a processor 202, software components 204, memory 206,audio processing components 208, audio amplifier(s) 210, speaker(s) 212,a network interface 214 including wireless interface(s) 216 and wiredinterface(s) 218, and microphone(s) 220. In one case, the playbackdevice 200 may not include the speaker(s) 212, but rather a speakerinterface for connecting the playback device 200 to external speakers.In another case, the playback device 200 may include neither thespeaker(s) 212 nor the audio amplifier(s) 210, but rather an audiointerface for connecting the playback device 200 to an external audioamplifier or audio-visual receiver.

In one example, the processor 202 may be a clock-driven computingcomponent configured to process input data according to instructionsstored in the memory 206. The memory 206 may be a tangiblecomputer-readable medium configured to store instructions executable bythe processor 202. For instance, the memory 206 may be data storage thatcan be loaded with one or more of the software components 204 executableby the processor 202 to achieve certain functions. In one example, thefunctions may involve the playback device 200 retrieving audio data froman audio source or another playback device. In another example, thefunctions may involve the playback device 200 sending audio data toanother device or playback device on a network. In yet another example,the functions may involve pairing of the playback device 200 with one ormore playback devices to create a multi-channel audio environment.

Certain functions may involve the playback device 200 synchronizingplayback of audio content with one or more other playback devices.During synchronous playback, a listener will preferably not be able toperceive time-delay differences between playback of the audio content bythe playback device 200 and the one or more other playback devices. U.S.Pat. No. 8,234,395 entitled, “System and method for synchronizingoperations among a plurality of independently clocked digital dataprocessing devices,” which is hereby incorporated by reference, providesin more detail some examples for audio playback synchronization amongplayback devices.

The memory 206 may further be configured to store data associated withthe playback device 200, such as one or more zones and/or zone groupsthe playback device 200 is a part of, audio sources accessible by theplayback device 200, or a playback queue that the playback device 200(or some other playback device) may be associated with. The data may bestored as one or more state variables that are periodically updated andused to describe the state of the playback device 200. The memory 206may also include the data associated with the state of the other devicesof the media system, and shared from time to time among the devices sothat one or more of the devices have the most recent data associatedwith the system. Other embodiments are also possible.

The audio processing components 208 may include one or moredigital-to-analog converters (DAC), an audio preprocessing component, anaudio enhancement component or a digital signal processor (DSP), and soon. In one embodiment, one or more of the audio processing components208 may be a subcomponent of the processor 202. In one example, audiocontent may be processed and/or intentionally altered by the audioprocessing components 208 to produce audio signals. The produced audiosignals may then be provided to the audio amplifier(s) 210 foramplification and playback through speaker(s) 212. Particularly, theaudio amplifier(s) 210 may include devices configured to amplify audiosignals to a level for driving one or more of the speakers 212. Thespeaker(s) 212 may include an individual transducer (e.g., a “driver”)or a complete speaker system involving an enclosure with one or moredrivers. A particular driver of the speaker(s) 212 may include, forexample, a subwoofer (e.g., for low frequencies), a mid-range driver(e.g., for middle frequencies), and/or a tweeter (e.g., for highfrequencies). In some cases, each transducer in the one or more speakers212 may be driven by an individual corresponding audio amplifier of theaudio amplifier(s) 210. In addition to producing analog signals forplayback by the playback device 200, the audio processing components 208may be configured to process audio content to be sent to one or moreother playback devices for playback.

Audio content to be processed and/or played back by the playback device200 may be received from an external source, such as via an audioline-in input connection (e.g., an auto-detecting 3.5 mm audio line-inconnection) or the network interface 214.

The network interface 214 may be configured to facilitate a data flowbetween the playback device 200 and one or more other devices on a datanetwork. As such, the playback device 200 may be configured to receiveaudio content over the data network from one or more other playbackdevices in communication with the playback device 200, network deviceswithin a local area network, or audio content sources over a wide areanetwork such as the Internet. In one example, the audio content andother signals transmitted and received by the playback device 200 may betransmitted in the form of digital packet data containing an InternetProtocol (IP)-based source address and IP-based destination addresses.In such a case, the network interface 214 may be configured to parse thedigital packet data such that the data destined for the playback device200 is properly received and processed by the playback device 200.

As shown, the network interface 214 may include wireless interface(s)216 and wired interface(s) 218. The wireless interface(s) 216 mayprovide network interface functions for the playback device 200 towirelessly communicate with other devices (e.g., other playbackdevice(s), speaker(s), receiver(s), network device(s), control device(s)within a data network the playback device 200 is associated with) inaccordance with a communication protocol (e.g., any wireless standardincluding IEEE 802.11a, 802.11b, 802.11g, 802.11n, 802.11ac, 802.15, 4Gmobile communication standard, and so on). The wired interface(s) 218may provide network interface functions for the playback device 200 tocommunicate over a wired connection with other devices in accordancewith a communication protocol (e.g., IEEE 802.3). While the networkinterface 214 shown in FIG. 2 includes both wireless interface(s) 216and wired interface(s) 218, the network interface 214 may in someembodiments include only wireless interface(s) or only wiredinterface(s).

The microphone(s) 220 may be arranged to detect sound in the environmentof the playback device 200. For instance, the microphone(s) may bemounted on an exterior wall of a housing of the playback device. Themicrophone(s) may be any type of microphone now known or later developedsuch as a condenser microphone, electret condenser microphone, or adynamic microphone. The microphone(s) may be sensitive to a portion ofthe frequency range of the speaker(s) 220. One or more of the speaker(s)220 may operate in reverse as the microphone(s) 220. In some aspects,the playback device 200 might not have microphone(s) 220.

In one example, the playback device 200 and one other playback devicemay be paired to play two separate audio components of audio content.For instance, playback device 200 may be configured to play a leftchannel audio component, while the other playback device may beconfigured to play a right channel audio component, thereby producing orenhancing a stereo effect of the audio content. The paired playbackdevices (also referred to as “bonded playback devices”) may further playaudio content in synchrony with other playback devices.

In another example, the playback device 200 may be sonicallyconsolidated with one or more other playback devices to form a single,consolidated playback device. A consolidated playback device may beconfigured to process and reproduce sound differently than anunconsolidated playback device or playback devices that are paired,because a consolidated playback device may have additional speakerdrivers through which audio content may be rendered. For instance, ifthe playback device 200 is a playback device designed to render lowfrequency range audio content (i.e. a subwoofer), the playback device200 may be consolidated with a playback device designed to render fullfrequency range audio content. In such a case, the full frequency rangeplayback device, when consolidated with the low frequency playbackdevice 200, may be configured to render only the mid and high frequencycomponents of audio content, while the low frequency range playbackdevice 200 renders the low frequency component of the audio content. Theconsolidated playback device may further be paired with a singleplayback device or yet another consolidated playback device.

By way of illustration, SONOS, Inc. presently offers (or has offered)for sale certain playback devices including a “PLAY:1,” “PLAY:3,”“PLAY:5,” “PLAYBAR,” “CONNECT:AMP,” “CONNECT,” and “SUB.” Any otherpast, present, and/or future playback devices may additionally oralternatively be used to implement the playback devices of exampleembodiments disclosed herein. Additionally, it is understood that aplayback device is not limited to the example illustrated in FIG. 2 orto the SONOS product offerings. For example, a playback device mayinclude a wired or wireless headphone. In another example, a playbackdevice may include or interact with a docking station for personalmobile media playback devices. In yet another example, a playback devicemay be integral to another device or component such as a television, alighting fixture, or some other device for indoor or outdoor use.

b. Example Playback Zone Configurations

Referring back to the media playback system 100 of FIG. 1, theenvironment may have one or more playback zones, each with one or moreplayback devices. The media playback system 100 may be established withone or more playback zones, after which one or more zones may be added,or removed to arrive at the example configuration shown in FIG. 1. Eachzone may be given a name according to a different room or space such asan office, bathroom, master bedroom, bedroom, kitchen, dining room,living room, and/or balcony. In one case, a single playback zone mayinclude multiple rooms or spaces. In another case, a single room orspace may include multiple playback zones.

As shown in FIG. 1, the balcony, dining room, kitchen, bathroom, office,and bedroom zones each have one playback device, while the living roomand master bedroom zones each have multiple playback devices. In theliving room zone, playback devices 104, 106, 108, and 110 may beconfigured to play audio content in synchrony as individual playbackdevices, as one or more bonded playback devices, as one or moreconsolidated playback devices, or any combination thereof. Similarly, inthe case of the master bedroom, playback devices 122 and 124 may beconfigured to play audio content in synchrony as individual playbackdevices, as a bonded playback device, or as a consolidated playbackdevice.

In one example, one or more playback zones in the environment of FIG. 1may each be playing different audio content. For instance, the user maybe grilling in the balcony zone and listening to hip hop music beingplayed by the playback device 102 while another user may be preparingfood in the kitchen zone and listening to classical music being playedby the playback device 114. In another example, a playback zone may playthe same audio content in synchrony with another playback zone. Forinstance, the user may be in the office zone where the playback device118 is playing the same rock music that is being playing by playbackdevice 102 in the balcony zone. In such a case, playback devices 102 and118 may be playing the rock music in synchrony such that the user mayseamlessly (or at least substantially seamlessly) enjoy the audiocontent that is being played out-loud while moving between differentplayback zones. Synchronization among playback zones may be achieved ina manner similar to that of synchronization among playback devices, asdescribed in previously referenced U.S. Pat. No. 8,234,395.

As suggested above, the zone configurations of the media playback system100 may be dynamically modified, and in some embodiments, the mediaplayback system 100 supports numerous configurations. For instance, if auser physically moves one or more playback devices to or from a zone,the media playback system 100 may be reconfigured to accommodate thechange(s). For instance, if the user physically moves the playbackdevice 102 from the balcony zone to the office zone, the office zone maynow include both the playback device 118 and the playback device 102.The playback device 102 may be paired or grouped with the office zoneand/or renamed if so desired via a control device such as the controldevices 126 and 128. On the other hand, if the one or more playbackdevices are moved to a particular area in the home environment that isnot already a playback zone, a new playback zone may be created for theparticular area.

Further, different playback zones of the media playback system 100 maybe dynamically combined into zone groups or split up into individualplayback zones. For instance, the dining room zone and the kitchen zone114 may be combined into a zone group for a dinner party such thatplayback devices 112 and 114 may render audio content in synchrony. Onthe other hand, the living room zone may be split into a television zoneincluding playback device 104, and a listening zone including playbackdevices 106, 108, and 110, if the user wishes to listen to music in theliving room space while another user wishes to watch television.

c. Example Control Devices

FIG. 3 shows a functional block diagram of an example control device 300that may be configured to be one or both of the control devices 126 and128 of the media playback system 100. As shown, the control device 300may include a processor 302, memory 304, a network interface 306, a userinterface 308, microphone(s) 310, and software components 312. In oneexample, the control device 300 may be a dedicated controller for themedia playback system 100. In another example, the control device 300may be a network device on which media playback system controllerapplication software may be installed, such as for example, an iPhone™,iPad™ or any other smart phone, tablet or network device (e.g., anetworked computer such as a PC or Mac™).

The processor 302 may be configured to perform functions relevant tofacilitating user access, control, and configuration of the mediaplayback system 100. The memory 304 may be data storage that can beloaded with one or more of the software components executable by theprocessor 302 to perform those functions. The memory 304 may also beconfigured to store the media playback system controller applicationsoftware and other data associated with the media playback system 100and the user.

In one example, the network interface 306 may be based on an industrystandard (e.g., infrared, radio, wired standards including IEEE 802.3,wireless standards including IEEE 802.11a, 802.11b, 802.11g, 802.11n,802.11ac, 802.15, 4G mobile communication standard, and so on). Thenetwork interface 306 may provide a means for the control device 300 tocommunicate with other devices in the media playback system 100. In oneexample, data and information (e.g., such as a state variable) may becommunicated between control device 300 and other devices via thenetwork interface 306. For instance, playback zone and zone groupconfigurations in the media playback system 100 may be received by thecontrol device 300 from a playback device or another network device, ortransmitted by the control device 300 to another playback device ornetwork device via the network interface 306. In some cases, the othernetwork device may be another control device.

Playback device control commands such as volume control and audioplayback control may also be communicated from the control device 300 toa playback device via the network interface 306. As suggested above,changes to configurations of the media playback system 100 may also beperformed by a user using the control device 300. The configurationchanges may include adding/removing one or more playback devices to/froma zone, adding/removing one or more zones to/from a zone group, forminga bonded or consolidated player, separating one or more playback devicesfrom a bonded or consolidated player, among others. Accordingly, thecontrol device 300 may sometimes be referred to as a controller, whetherthe control device 300 is a dedicated controller or a network device onwhich media playback system controller application software isinstalled.

Control device 300 may include microphone(s) 310. Microphone(s) 310 maybe arranged to detect sound in the environment of the control device300. Microphone(s) 310 may be any type of microphone now known or laterdeveloped such as a condenser microphone, electret condenser microphone,or a dynamic microphone. The microphone(s) may be sensitive to a portionof a frequency range. Two or more microphones 310 may be arranged tocapture location information of an audio source (e.g., voice, audiblesound) and/or to assist in filtering background noise.

The user interface 308 of the control device 300 may be configured tofacilitate user access and control of the media playback system 100, byproviding a controller interface such as the controller interface 400shown in FIG. 4. The controller interface 400 includes a playbackcontrol region 410, a playback zone region 420, a playback status region430, a playback queue region 440, and an audio content sources region450. The user interface 400 as shown is just one example of a userinterface that may be provided on a network device such as the controldevice 300 of FIG. 3 (and/or the control devices 126 and 128 of FIG. 1)and accessed by users to control a media playback system such as themedia playback system 100. Other user interfaces of varying formats,styles, and interactive sequences may alternatively be implemented onone or more network devices to provide comparable control access to amedia playback system.

The playback control region 410 may include selectable (e.g., by way oftouch or by using a cursor) icons to cause playback devices in aselected playback zone or zone group to play or pause, fast forward,rewind, skip to next, skip to previous, enter/exit shuffle mode,enter/exit repeat mode, enter/exit cross fade mode. The playback controlregion 410 may also include selectable icons to modify equalizationsettings, and playback volume, among other possibilities.

The playback zone region 420 may include representations of playbackzones within the media playback system 100. In some embodiments, thegraphical representations of playback zones may be selectable to bringup additional selectable icons to manage or configure the playback zonesin the media playback system, such as a creation of bonded zones,creation of zone groups, separation of zone groups, and renaming of zonegroups, among other possibilities.

For example, as shown, a “group” icon may be provided within each of thegraphical representations of playback zones. The “group” icon providedwithin a graphical representation of a particular zone may be selectableto bring up options to select one or more other zones in the mediaplayback system to be grouped with the particular zone. Once grouped,playback devices in the zones that have been grouped with the particularzone will be configured to play audio content in synchrony with theplayback device(s) in the particular zone. Analogously, a “group” iconmay be provided within a graphical representation of a zone group. Inthis case, the “group” icon may be selectable to bring up options todeselect one or more zones in the zone group to be removed from the zonegroup. Other interactions and implementations for grouping andungrouping zones via a user interface such as the user interface 400 arealso possible. The representations of playback zones in the playbackzone region 420 may be dynamically updated as playback zone or zonegroup configurations are modified.

The playback status region 430 may include graphical representations ofaudio content that is presently being played, previously played, orscheduled to play next in the selected playback zone or zone group. Theselected playback zone or zone group may be visually distinguished onthe user interface, such as within the playback zone region 420 and/orthe playback status region 430. The graphical representations mayinclude track title, artist name, album name, album year, track length,and other relevant information that may be useful for the user to knowwhen controlling the media playback system via the user interface 400.

The playback queue region 440 may include graphical representations ofaudio content in a playback queue associated with the selected playbackzone or zone group. In some embodiments, each playback zone or zonegroup may be associated with a playback queue containing informationcorresponding to zero or more audio items for playback by the playbackzone or zone group. For instance, each audio item in the playback queuemay comprise a uniform resource identifier (URI), a uniform resourcelocator (URL) or some other identifier that may be used by a playbackdevice in the playback zone or zone group to find and/or retrieve theaudio item from a local audio content source or a networked audiocontent source, possibly for playback by the playback device.

In one example, a playlist may be added to a playback queue, in whichcase information corresponding to each audio item in the playlist may beadded to the playback queue. In another example, audio items in aplayback queue may be saved as a playlist. In a further example, aplayback queue may be empty, or populated but “not in use” when theplayback zone or zone group is playing continuously streaming audiocontent, such as Internet radio that may continue to play untilotherwise stopped, rather than discrete audio items that have playbackdurations. In an alternative embodiment, a playback queue can includeInternet radio and/or other streaming audio content items and be “inuse” when the playback zone or zone group is playing those items. Otherexamples are also possible.

When playback zones or zone groups are “grouped” or “ungrouped,”playback queues associated with the affected playback zones or zonegroups may be cleared or re-associated. For example, if a first playbackzone including a first playback queue is grouped with a second playbackzone including a second playback queue, the established zone group mayhave an associated playback queue that is initially empty, that containsaudio items from the first playback queue (such as if the secondplayback zone was added to the first playback zone), that contains audioitems from the second playback queue (such as if the first playback zonewas added to the second playback zone), or a combination of audio itemsfrom both the first and second playback queues. Subsequently, if theestablished zone group is ungrouped, the resulting first playback zonemay be re-associated with the previous first playback queue, or beassociated with a new playback queue that is empty or contains audioitems from the playback queue associated with the established zone groupbefore the established zone group was ungrouped. Similarly, theresulting second playback zone may be re-associated with the previoussecond playback queue, or be associated with a new playback queue thatis empty, or contains audio items from the playback queue associatedwith the established zone group before the established zone group wasungrouped. Other examples are also possible.

Referring back to the user interface 400 of FIG. 4, the graphicalrepresentations of audio content in the playback queue region 440 mayinclude track titles, artist names, track lengths, and other relevantinformation associated with the audio content in the playback queue. Inone example, graphical representations of audio content may beselectable to bring up additional selectable icons to manage and/ormanipulate the playback queue and/or audio content represented in theplayback queue. For instance, a represented audio content may be removedfrom the playback queue, moved to a different position within theplayback queue, or selected to be played immediately, or after anycurrently playing audio content, among other possibilities. A playbackqueue associated with a playback zone or zone group may be stored in amemory on one or more playback devices in the playback zone or zonegroup, on a playback device that is not in the playback zone or zonegroup, and/or some other designated device.

The audio content sources region 450 may include graphicalrepresentations of selectable audio content sources from which audiocontent may be retrieved and played by the selected playback zone orzone group. Discussions pertaining to audio content sources may be foundin the following section.

d. Example Audio Content Sources

As indicated previously, one or more playback devices in a zone or zonegroup may be configured to retrieve for playback audio content (e.g.according to a corresponding URI or URL for the audio content) from avariety of available audio content sources. In one example, audiocontent may be retrieved by a playback device directly from acorresponding audio content source (e.g., a line-in connection). Inanother example, audio content may be provided to a playback device overa network via one or more other playback devices or network devices.

Example audio content sources may include a memory of one or moreplayback devices in a media playback system such as the media playbacksystem 100 of FIG. 1, local music libraries on one or more networkdevices (such as a control device, a network-enabled personal computer,or a networked-attached storage (NAS), for example), streaming audioservices providing audio content via the Internet (e.g., the cloud), oraudio sources connected to the media playback system via a line-in inputconnection on a playback device or network devise, among otherpossibilities.

In some embodiments, audio content sources may be regularly added orremoved from a media playback system such as the media playback system100 of FIG. 1. In one example, an indexing of audio items may beperformed whenever one or more audio content sources are added, removedor updated. Indexing of audio items may involve scanning foridentifiable audio items in all folders/directory shared over a networkaccessible by playback devices in the media playback system, andgenerating or updating an audio content database containing metadata(e.g., title, artist, album, track length, among others) and otherassociated information, such as a URI or URL for each identifiable audioitem found. Other examples for managing and maintaining audio contentsources may also be possible.

The above discussions relating to playback devices, controller devices,playback zone configurations, and media content sources provide onlysome examples of operating environments within which functions andmethods described below may be implemented. Other operating environmentsand configurations of media playback systems, playback devices, andnetwork devices not explicitly described herein may also be applicableand suitable for implementation of the functions and methods.

e. Example Plurality of Networked Devices

FIG. 5 shows an example plurality of devices 500 that may be configuredto provide an audio playback experience based on voice control. Onehaving ordinary skill in the art will appreciate that the devices shownin FIG. 5 are for illustrative purposes only, and variations includingdifferent and/or additional devices may be possible. As shown, theplurality of devices 500 includes computing devices 504, 506, and 508;network microphone devices (NMDs) 512, 514, and 516; playback devices(PBDs) 532, 534, 536, and 538; and a controller device (CR) 522.

Each of the plurality of devices 500 may be network-capable devices thatcan establish communication with one or more other devices in theplurality of devices according to one or more network protocols, such asNFC, Bluetooth, Ethernet, and IEEE 802.11, among other examples, overone or more types of networks, such as wide area networks (WAN), localarea networks (LAN), and personal area networks (PAN), among otherpossibilities.

As shown, the computing devices 504, 506, and 508 may be part of a cloudnetwork 502. The cloud network 502 may include additional computingdevices. In one example, the computing devices 504, 506, and 508 may bedifferent servers. In another example, two or more of the computingdevices 504, 506, and 508 may be modules of a single server.Analogously, each of the computing device 504, 506, and 508 may includeone or more modules or servers. For ease of illustration purposesherein, each of the computing devices 504, 506, and 508 may beconfigured to perform particular functions within the cloud network 502.For instance, computing device 508 may be a source of audio content fora streaming music service.

As shown, the computing device 504 may be configured to interface withNMDs 512, 514, and 516 via communication path 542. NMDs 512, 514, and516 may be components of one or more “Smart Home” systems. In one case,NMDs 512, 514, and 516 may be physically distributed throughout ahousehold, similar to the distribution of devices shown in FIG. 1. Inanother case, two or more of the NMDs 512, 514, and 516 may bephysically positioned within relative close proximity of one another.Communication path 542 may comprise one or more types of networks, suchas a WAN including the Internet, LAN, and/or PAN, among otherpossibilities.

In one example, one or more of the NMDs 512, 514, and 516 may be devicesconfigured primarily for audio detection. In another example, one ormore of the NMDs 512, 514, and 516 may be components of devices havingvarious primary utilities. For instance, as discussed above inconnection to FIGS. 2 and 3, one or more of NMDs 512, 514, and 516 maybe the microphone(s) 220 of playback device 200 or the microphone(s) 310of network device 300. Further, in some cases, one or more of NMDs 512,514, and 516 may be the playback device 200 or network device 300. In anexample, one or more of NMDs 512, 514, and/or 516 may include multiplemicrophones arranged in a microphone array.

As shown, the computing device 506 may be configured to interface withCR 522 and PBDs 532, 534, 536, and 538 via communication path 544. Inone example, CR 522 may be a network device such as the network device200 of FIG. 2. Accordingly, CR 522 may be configured to provide thecontroller interface 400 of FIG. 4. Similarly, PBDs 532, 534, 536, and538 may be playback devices such as the playback device 300 of FIG. 3.As such, PBDs 532, 534, 536, and 538 may be physically distributedthroughout a household as shown in FIG. 1. For illustration purposes,PBDs 536 and 538 may be part of a bonded zone 530, while PBDs 532 and534 may be part of their own respective zones. As described above, thePBDs 532, 534, 536, and 538 may be dynamically bonded, grouped,unbonded, and ungrouped. Communication path 544 may comprise one or moretypes of networks, such as a WAN including the Internet, LAN, and/orPAN, among other possibilities.

In one example, as with NMDs 512, 514, and 516, CR 522 and PBDs 532,534, 536, and 538 may also be components of one or more “Smart Home”systems. In one case, PBDs 532, 534, 536, and 538 may be distributedthroughout the same household as the NMDs 512, 514, and 516. Further, assuggested above, one or more of PBDs 532, 534, 536, and 538 may be oneor more of NMDs 512, 514, and 516.

The NMDs 512, 514, and 516 may be part of a local area network, and thecommunication path 542 may include an access point that links the localarea network of the NMDs 512, 514, and 516 to the computing device 504over a WAN (communication path not shown). Likewise, each of the NMDs512, 514, and 516 may communicate with each other via such an accesspoint.

Similarly, CR 522 and PBDs 532, 534, 536, and 538 may be part of a localarea network and/or a local playback network as discussed in previoussections, and the communication path 544 may include an access pointthat links the local area network and/or local playback network of CR522 and PBDs 532, 534, 536, and 538 to the computing device 506 over aWAN. As such, each of the CR 522 and PBDs 532, 534, 536, and 538 mayalso communicate with each over such an access point.

In one example, communication paths 542 and 544 may comprise the sameaccess point. In an example, each of the NMDs 512, 514, and 516, CR 522,and PBDs 532, 534, 536, and 538 may access the cloud network 502 via thesame access point for a household.

As shown in FIG. 5, each of the NMDs 512, 514, and 516, CR 522, and PBDs532, 534, 536, and 538 may also directly communicate with one or more ofthe other devices via communication means 546. Communication means 546as described herein may involve one or more forms of communicationbetween the devices, according to one or more network protocols, overone or more types of networks, and/or may involve communication via oneor more other network devices. For instance, communication means 546 mayinclude one or more of for example, Bluetooth™ (IEEE 802.15), NFC,Wireless direct, and/or Proprietary wireless, among other possibilities.

In one example, CR 522 may communicate with NMD 512 over Bluetooth™, andcommunicate with PBD 534 over another local area network. In anotherexample, NMD 514 may communicate with CR 522 over another local areanetwork, and communicate with PBD 536 over Bluetooth. In a furtherexample, each of the PBDs 532, 534, 536, and 538 may communicate witheach other according to a spanning tree protocol over a local playbacknetwork, while each communicating with CR 522 over a local area network,different from the local playback network. Other examples are alsopossible.

In some cases, communication means between the NMDs 512, 514, and 516,CR 522, and PBDs 532, 534, 536, and 538 may change depending on types ofcommunication between the devices, network conditions, and/or latencydemands. For instance, communication means 546 may be used when NMD 516is first introduced to the household with the PBDs 532, 534, 536, and538. In one case, the NMD 516 may transmit identification informationcorresponding to the NMD 516 to PBD 538 via NFC, and PBD 538 may inresponse, transmit local area network information to NMD 516 via NFC (orsome other form of communication). However, once NMD 516 has beenconfigured within the household, communication means between NMD 516 andPBD 538 may change. For instance, NMD 516 may subsequently communicatewith PBD 538 via communication path 542, the cloud network 502, andcommunication path 544. In another example, the NMDs and PBDs may nevercommunicate via local communications means 546. In a further example,the NMDs and PBDs may communicate primarily via local communicationsmeans 546. Other examples are also possible.

In an illustrative example, NMDs 512, 514, and 516 may be configured toreceive voice inputs to control PBDs 532, 534, 536, and 538. Theavailable control commands may include any media playback systemcontrols previously discussed, such as playback volume control, playbacktransport controls, music source selection, and grouping, among otherpossibilities. In one instance, NMD 512 may receive a voice input tocontrol one or more of the PBDs 532, 534, 536, and 538. In response toreceiving the voice input, NMD 512 may transmit via communication path542, the voice input to computing device 504 for processing. In oneexample, the computing device 504 may convert the voice input to anequivalent text command, and parse the text command to identify acommand. Computing device 504 may then subsequently transmit the textcommand to the computing device 506. In another example, the computingdevice 504 may convert the voice input to an equivalent text command,and then subsequently transmit the text command to the computing device506. The computing device 506 may then parse the text command toidentify one or more playback commands.

For instance, if the text command is “Play ‘Track 1’ by ‘Artist 1’ from‘Streaming Service 1’ in ‘Zone 1’,” The computing device 506 mayidentify (i) a URL for “Track 1” by “Artist 1” available from “StreamingService 1,” and (ii) at least one playback device in “Zone 1.” In thisexample, the URL for “Track 1” by “Artist 1” from “Streaming Service 1”may be a URL pointing to computing device 508, and “Zone 1” may be thebonded zone 530. As such, upon identifying the URL and one or both ofPBDs 536 and 538, the computing device 506 may transmit viacommunication path 544 to one or both of PBDs 536 and 538, theidentified URL for playback. One or both of PBDs 536 and 538 mayresponsively retrieve audio content from the computing device 508according to the received URL, and begin playing “Track 1” by “Artist 1”from “Streaming Service 1.”

One having ordinary skill in the art will appreciate that the above isjust one illustrative example, and that other implementations are alsopossible. In one case, operations performed by one or more of theplurality of devices 500, as described above, may be performed by one ormore other devices in the plurality of device 500. For instance, theconversion from voice input to the text command may be alternatively,partially, or wholly performed by another device or devices, such as NMD512, computing device 506, PBD 536, and/or PBD 538. Analogously, theidentification of the URL may be alternatively, partially, or whollyperformed by another device or devices, such as NMD 512, computingdevice 504, PBD 536, and/or PBD 538.

f. Example Network Microphone Device

FIG. 6 shows a function block diagram of an example network microphonedevice 600 that may be configured to be one or more of NMDs 512, 514,and 516 of FIG. 5. As shown, the network microphone device 600 includesa processor 602, memory 604, a microphone array 606, a network interface608, a user interface 610, software components 612, and speaker(s) 614.One having ordinary skill in the art will appreciate that other networkmicrophone device configurations and arrangements are also possible. Forinstance, network microphone devices may alternatively exclude thespeaker(s) 614 or have a single microphone instead of microphone array606.

The processor 602 may include one or more processors and/or controllers,which may take the form of a general or special-purpose processor orcontroller. For instance, the processing unit 602 may includemicroprocessors, microcontrollers, application-specific integratedcircuits, digital signal processors, and the like. The memory 604 may bedata storage that can be loaded with one or more of the softwarecomponents executable by the processor 602 to perform those functions.Accordingly, memory 604 may comprise one or more non-transitorycomputer-readable storage mediums, examples of which may includevolatile storage mediums such as random access memory, registers, cache,etc. and non-volatile storage mediums such as read-only memory, ahard-disk drive, a solid-state drive, flash memory, and/or anoptical-storage device, among other possibilities.

The microphone array 606 may be a plurality of microphones arranged todetect sound in the environment of the network microphone device 600.Microphone array 606 may include any type of microphone now known orlater developed such as a condenser microphone, electret condensermicrophone, or a dynamic microphone, among other possibilities. In oneexample, the microphone array may be arranged to detect audio from oneor more directions relative to the network microphone device. Themicrophone array 606 may be sensitive to a portion of a frequency range.In one example, a first subset of the microphone array 606 may besensitive to a first frequency range, while a second subset of themicrophone array may be sensitive to a second frequency range. Themicrophone array 606 may further be arranged to capture locationinformation of an audio source (e.g., voice, audible sound) and/or toassist in filtering background noise. Notably, in some embodiments themicrophone array may consist of only a single microphone, rather than aplurality of microphones.

The network interface 608 may be configured to facilitate wirelessand/or wired communication between various network devices, such as, inreference to FIG. 5, CR 522, PBDs 532-538, computing device 504-508 incloud network 502, and other network microphone devices, among otherpossibilities. As such, network interface 608 may take any suitable formfor carrying out these functions, examples of which may include anEthernet interface, a serial bus interface (e.g., FireWire, USB 2.0,etc.), a chipset and antenna adapted to facilitate wirelesscommunication, and/or any other interface that provides for wired and/orwireless communication. In one example, the network interface 608 may bebased on an industry standard (e.g., infrared, radio, wired standardsincluding IEEE 802.3, wireless standards including IEEE 802.11a,802.11b, 802.11g, 802.11n, 802.11ac, 802.15, 4G mobile communicationstandard, and so on).

The user interface 610 of the network microphone device 600 may beconfigured to facilitate user interactions with the network microphonedevice. In one example, the user interface 608 may include one or moreof physical buttons, graphical interfaces provided on touch sensitivescreen(s) and/or surface(s), among other possibilities, for a user todirectly provide input to the network microphone device 600. The userinterface 610 may further include one or more of lights and thespeaker(s) 614 to provide visual and/or audio feedback to a user. In oneexample, the network microphone device 600 may further be configured toplayback audio content via the speaker(s) 614.

III. Example Method of Playing an Audio Response

As discussed above, embodiments described herein may involve playingback an audio response to a voice command. FIG. 7 shows an example flowdiagram 700 of a method for playing an audio response, and FIG. 8 showsan example flow diagram 800 presenting aspects of blocks 708 and 712 ofFIG. 7 in more detail. Methods 700 and 800 present embodiments ofmethods that can be implemented within an operating environmentinvolving, for example, the media playback system 100 of FIG. 1, one ormore of the playback device 200 of FIG. 2, one or more of the controldevice 300 of FIG. 3, and one or more of the plurality of devices insystem 500 of FIG. 5. Methods 700 and 800 may include one or moreoperations, functions, or actions. Although the blocks in FIGS. 7 and 8are illustrated in sequential order, these blocks may also be performedin parallel, and/or in a different order than those described herein.Also, the various blocks may be combined into fewer blocks, divided intoadditional blocks, and/or removed based upon the desired implementation.

In addition, for the methods 700 and 800 and other processes and methodsdisclosed herein, the flow diagrams show functionality and operation ofone possible implementation of present embodiments. In this regard, eachblock may represent a module, a segment, or a portion of program code,which includes one or more instructions executable by a processor forimplementing specific logical functions or steps in the process. Theprogram code may be stored on any type of computer readable medium, forexample, such as a storage device including a disk or hard drive. Thecomputer readable medium may include non-transitory computer readablemedium, for example, such as computer-readable media that stores datafor short periods of time like register memory, processor cache andRandom Access Memory (RAM). The computer readable medium may alsoinclude non-transitory media, such as secondary or persistent long termstorage, like read only memory (ROM), optical or magnetic disks,compact-disc read only memory (CD-ROM), for example. The computerreadable media may also be any other volatile or non-volatile storagesystems. The computer readable medium may be considered a computerreadable storage medium, for example, or a tangible storage device. Inaddition, for the methods 700 and 800 and other processes and methodsdisclosed herein, each block in FIGS. 7 and 8 may represent circuitrythat is wired to perform the specific logical functions in the process.

As discussed above, FIG. 7 depicts a flow diagram of a method 700 forplaying an audio response. At block 702 of method 700, a networkmicrophone device 512, 514, 516 may be assigned (e.g., paired or bonded)to a zone comprising one or more playback devices such as playbackdevices 532, 534, 536, 538. Network microphone devices may be assignedto a zone as described in the application identified as U.S. ProvisionalApplication No. 62/298,410 entitled Default Playback Device(s), thecontents of which is hereby incorporated by reference in its entirety.At block 704, a user may provide a voice command to one or more devicesof the plurality of devices 500 which contain a microphone (e.g., NMD512, 514, 516, PBD 532, 534, 536, 538, CR 522). The one or moremicrophone devices may pass the voice command to other devices of theplurality or system of devices 500 (e.g., one or more computing devices504, 506, 508 in cloud network 502, NMD 512, 514, 516, PBD 532, 534,536, 538) for processing. At block 706, the one or more processingdevices may process the voice command and return a response to be playedback. The response to be played back may be returned in the form of avoice response message comprising the audio clip to be played, text tobe converted to speech, and/or an identifier (e.g., URL or URI) of apre-recorded response to be played that may be stored locally on adevice of system 500). The response may be returned via one or morecommunication paths 542, 544, 546.

At block 708, system 500 may select one or more devices or one or morezones to play the voice response. The determination or selection ofwhich one or more devices or zones to play the voice response may dependon a variety of factors including context information, pairinginformation, and/or user settings. Block 708 will be discussed in moredetail with respect to FIG. 8.

At block 710, system 500 may receive the voice response messageindicating the response to play. Block 710 may occur before, inconjunction with, or after block 708. At block 712, the system 500 maycause the one or more device(s) selected in block 708 to play the voiceresponse. The default zone may be grouped for synchronous playback withanother zone and playback of the audio response may be adjusted based onthe grouping which will be described in more detail with respect to FIG.8.

FIG. 8 shows an illustrative flow diagram of selecting one or more zonesor devices (e.g., individual device, zone comprising one or moredevices) to play the voice response to the voice command according toaspects described herein. At block 802, a computing device (e.g., NMD,playback device, control device, computing device) can determine whetherto use a predetermined response device (e.g., default PBD) or zone(e.g., default zone) or dynamically select a response device or zone toplay the voice response. A predetermined response device or zone may bedetermined during initial system setup or configuration (e.g.,assignment, pairing, bonding) when adding new devices to system 500. Forexample, initial setup or configuration may be to add a new playbackdevice to system 500 and/or to add or assign a new NMD to a zone (e.g.,default zone). The predetermined response device may be selected by theuser and/or automatically selected by the system 500. A response deviceor zone may be dynamically selected based on various information such ascontext information (e.g., location information, orientationinformation, presence information) which may include pairing information(e.g., pairing dependent on location of a user, pairing dependent onlocation of microphone device), and/or capability information (e.g.,available sensors, available devices, available networks, availableprocessing power).

The decision of whether to use a predetermined device or to dynamicallyselect an audio response device may be based on the capabilities ofsystem 500 to dynamically select a response device and/or for anefficient determination of the response device. Certain devices and/orhardware may be used in order to make a dynamic selection, and if thecertain devices and/or hardware are not available to participate in thedynamic selection, system 500 may default to using a predeterminedresponse device. For example, the system 500 might not have thecapability to determine location information related to the user'sposition and/or the position of a microphone device (e.g., NMD, PBD, CR)receiving or having responsibility of processing a voice command spokenby the user, and/or the system 500 might not have previously determinedlocation information of devices in system 500. In such an example, adetermination to use a predetermined device may be the most expedientavenue to provide the user with an immediate acknowledgement or responseto the voice command. As another example, the system 500 may lackhardware such as a sensor, camera, or other input device which can beused to determine context information (e.g., location information,presence information, etc.). In this instance, the system 500 may decideto use a predetermined response device (e.g., default zone, defaultplayback device). As yet another example, the system 500 may have onlyone zone in which case the system can automatically determine to set theone zone as the default zone and/or to use the one zone for all audioresponses. Similarly, the system 500 may have only a single PBD. In viewof this, the system may select to use the single PBD as the defaultdevice and/or predetermined audio response device.

In some instances, location information may be determined or processedby transmitting information detected by devices in the system 500 tocloud network 502 for processing via communication paths 542, 544,and/or 546. Cloud network 502 may possess more processing power than thedevices NMD 512, 514, 516, CR 522, and PBD 532, 534, 536, 538. In theevent the cloud network 502 is not available, the system 500 may use oneof the predetermined devices and may defer to a user's preferencesstored in a user profile for the specific predetermined device (e.g.,default response device) to speak or play an audio response. In someaspects, the location information may be processed locally by one ormore of the NMDs, CRs, PBDs.

If the system 500 has determined at block 802 to select one or morepredetermined response devices or zones to play the audio response tothe voice command, method 800 may proceed to block 804. If the system500 has determined at block 802 to dynamically select one or moreresponse devices to play the audio response, method 800 may proceed toblock 806.

At block 804, a processing device (e.g., any of the plurality of devices500) may determine to select a predetermined zone or one or morepredetermined devices. For example, a user may designate one or moredevices or one or more zones to play the audio response to any voicecommand during initial setup or configuration (e.g., assignment, pairingor bonding), and the user designation may be saved as a default settingin a user profile associated with the user. The user profile may bestored in any device of the plurality of devices 500. In response toreceiving a voice command, the receiving device or any other device insystem 500 may retrieve the user profile stored locally or in anotherdevice to determine which predetermined speaker(s) or zone(s) ofplayback device(s) to use as the device(s) to play the audio response tothe voice command.

In one example, a network microphone device may include a speaker, andthe speaker of the networked microphone device may be used to play thevoice response.

In another aspect, a speaker in an audio response device (e.g., NMD(s),playback device(s), control device) may be configured during initialsetup with the NMD to play audio responses. For example, a user mayselect a particular device in the default zone to be a default responsedevice. In other instances, the default zone may include one playbackdevice, and the one playback device may be identified as the defaultresponse device. In some other instances, the default zone may includemore than one playback device, and a user may select one of the playbackdevices or two or more of the playback devices as the default responsedevice.

In some aspects, the zone to playback an audio response may be a bondedzone (e.g., bonded zone 530) including two or more playback devices(e.g., PBDs 536, 538). In a bonded zone, the audio response may beplayed by one or more the playback devices in the bonded zone, and whichof the playback devices of the bonded zone play the audio may depend onthe type of playback devices. For example, the bonded zone may include asurround sound system where playback devices may be placed at differentlocations around a user and be responsible for playing one or morechannels of audio. In this arrangement, the playback devices may beplaced both in front of and behind the user. Based on the orientation ofthe user to the PBDs, the PBD(s) in front of the user may be selected toplay the audio response.

In some aspects, the NMD may be usually stationary such that a relativelocation may be configured during initial setup. In one aspect, a usermay identify a response device nearest to the NMD. The response devicemay be part of the default zone or another zone. In another aspect, theresponse device nearest to the NMD may be identified automatically bynearby response devices and/or the NMD. For example, proximity detectionmay be performed using the microphone(s) of the nearby response devicesand/or NMD. For example, one or more playback devices in the defaultzone may emit or play a predetermined tone. The predetermined tone maybe played at the same power level or volume at each playback device. Themicrophone(s) of the NMD may receive the test tone and determine basedon highest power level which playback device is closest to the NMD.

In some aspects, the default response device may be determined based onorientation of the playback devices relative to the NMD. For example,playback devices which are facing towards the NMD may have higherpriority to be the default response device over a playback device whichis physically nearer to the NMD. Similar to using a test tone todetermine a nearest playback device, a test tone picked up by themicrophone(s) of the NMD may be used to determine orientation ofplayback devices relative to the NMD. The NMD may pick up or detect atone played from a playback device positioned so that speakers facingtowards the NMD detected more loudly or at a higher power than a speakerdirected away from the NMD. Other techniques for proximity or locationdetermination are possible including those described herein.

As another example, all playback devices may be selected to play theaudio response to the voice command. For example, all playback devicesin a default zone (e.g., zone to which the NMD is assigned) may play anyor all audio responses which are responsive to an audio or voicecommand.

After determining one or more predetermined response devices, the methodmay proceed to block 808 and determine whether the default zone isgrouped with another zone. The default zone may be grouped with anotherzone for synchronous playback of media items (e.g., audio content, videoclips, radio).

Returning to block 806, the system 500 may dynamically select one ormore response devices or one or more zones to play the audio response tothe audio command. The dynamically selected audio response device(s) maybe selected based on pairing information (e.g., NMD dynamic pairingbased on user location and/or device location) and/or contextinformation (e.g., orientation of user, context at time of the voicecommand, location information). The pairing information may be dependenton the context information. The dynamically selected device(s) may beselected independent of pairing information. The zone or device selectedfor the response may be the same or different from the device or zone towhich the NMD is assigned.

As discussed above, a NMD may be paired or assigned to a zone or adevice. The dynamically selected audio response device may be the deviceto which a NMD is paired where the pairing between the NMD and anotherdevice may change dynamically based on (e.g., change in response to achange in) a user's location which can be determined at any time, inresponse to a particular event (e.g., user requests a particular zone)and/or a location of the NMD relative to another device (e.g., NMD, PBD,CR). For example, at the time voice command is received or at the timethe audio response is to be played to the user. The user's location maybe determined using one or more of the plurality of devices 500 andvarious techniques. For example, microphones in two or more differentdevices of the plurality of devices 500 may be used in determining thelocation of the user relative to the two or more different devices.

Two or more different devices may be listening for a user's voice anddetermine a closest speaker by comparing a magnitude (e.g., volume,power level) of the user's voice and/or determining a delay or timingdifference between when each of two or more different devices receivesthe user's voice. As an example, a first device (e.g., PBD 532) and asecond device (e.g., PBD 534) may be located in the same zone as theuser, and the microphones of these playback devices may be able topickup the user's voice command. The devices may be able to recordand/or analyze the user's speech or voice when the user speaks the voicecommand. The recording may be done by any of the devices 500 whichinclude a microphone (e.g., NMD, PBD CR), and the analysis may be doneby any of the devices of the plurality of devices 500. The analysis mayinclude receiving the recorded user's speech or voice from one or moreother devices and comparing based on timestamp and/or by spoken wordand/or by syllable of the user's speech or other characteristic of therecording. In some aspects, the comparison may be performed after eachword, syllable, and/or phrase and need not wait until the user hascompleted speaking the voice command.

The delay or timing difference between when different devices receivethe sound of a user's voice may be indicative of a relative distancebetween the user and the receiving devices since devices closer to theuser may receive the sound of the user's voice before devices furtheraway from the user. For example, the two or more devices can eachtimestamp when certain sound characteristics or sound markers occur inthe voice input and compare the timestamp to determine which of the twoor more devices received the sound of the voice first. The two or moredevices may maintain synchronized clocks and can directly comparetimestamps of the sound characteristics to determine which devicereceived the sound characteristic first. In some aspects, the two ormore devices may determine a time difference between their respectiveclocks and compare the difference of the timestamps while subtractingthe time difference between their respective clocks.

The recording or analysis of the user's speech may be performed by oneor more devices and may be initiated by another device. For example, aNMD or other device containing a microphone (e.g., CR, PBD) may alwaysbe listening for a voice command, and when the NMD or other microphonedevice detects that the user has started speaking a command, the NMD mayalert other microphone devices (e.g., NMD, CR, PBD) to begin receiving,recording, and/or analyzing the user's voice command by transmitting analert (e.g., message) over a communication connection (e.g., 542, 544,546) to other microphone devices which may or might not be in the samezone as the NMD.

Similarly, the analysis of the user's speech may be used to determine auser's orientation relative to the speaker. For example, a user may befacing towards a first speaker and have a second speaker directed to theuser's back. In some instances, the system 500 may select the firstspeaker to play the audio response even though the user is closer to thesecond speaker since it may be desirable to have a speaker in front theof user play the audio response rather than a speaker behind the user. Amicrophone in a device (e.g., PBD, CR, NMD) in front of the user maypick up or detect a user's voice more loudly or at a higher power than amicrophone in a device (e.g., PBD, CR, NMD) to the side or behind theuser.

In some aspects, location information can include location of themicrophone device (e.g., NMD, PBD, CR) relative to a speaker. Forexample, one or more playback devices in the default zone may emit orplay a predetermined tone. The predetermined tone may be played at thesame power level or volume at each playback device and/or may be withinor outside a human hearing range (e.g., approximately 20 Hz to 20 kHz).The microphone(s) of the NMD may receive the test tone and determinebased on highest power level which playback device is closest to theNMD.

In some aspects, NMD 512 may include a microphone array 606. Using themicrophone array 606, the NMD may be able to generally identifydirection, distance, and/or orientation of the user by determining whichmicrophone(s) of microphone array 606 detects the loudest volume orhighest power of the user's voice. In some aspects, the system or NMD500 may have previously determined relative positions of playbackdevices and/or been programmed with relative positions of the playbackdevices, and the NMD may be able to use the microphones of the playbackdevices in combination with its own microphone(s) to determine generaldirection, distance, and/or orientation of a user using the user's voicewhen the voice command is spoken. Each determination of the user'slocation may be stored in one or more of the devices 500 and may bereferenced in response to later voice commands for a quickerdetermination of a user's location and/or orientation.

Another technique for determining the location of the user may be to useinterference and/or strength of wireless signals in an environment(e.g., location, zone). In some aspects, the location of the user may bedetermined based on wireless signals in an environment where the userand the PBD(s) and/or the NMD are located. The wireless signals mayinclude wireless communication signals according to any wirelesscommunication protocol such as an industry standard (e.g., IEEE 802.11a,802.11b, 802.11g, 802.11n, 802.11ac, 802.15, Bluetooth, 4G mobilecommunication standard, LTE mobile communication standard, Bluetooth®,Near-Field Communications (NFC), etc.). The user may have a device(e.g., computing device, mobile device, wearable, tablet, etc.) whichincludes a wireless interface to transmit wireless communication signalsaccording to a wireless communication protocol. The wireless signalsemitted from the device may cause interference with other wirelesssignals present in the environment where the user is located. Based onthe strength of the interference with other wireless signals, otherdevices which can detect the wireless signals and interference candetermine a location of the user.

To determine the location of the user based on the strength of wirelesssignals in an environment, the user's device may measure the signalstrength (e.g., received signal strength indicator (RSSI)) of a signalemitted by any wireless device in the environment with the user andidentify the device with the strongest signal strength as the devicenearest the user. Similarly, wireless device(s) (e.g., PBD, NMD,wireless router, beacon, NFC) in the environment can measure the signalstrength of a wireless signal emitted by the user's device and transmitand/or collect signal strength measurements of the user device'swireless signal by other devices. The signal strength measurements maybe collected and/or compared by any device in system 500, and the devicewith the strongest signal strength measurement can be identified as thedevice nearest to the user.

In some aspects, the NMD may become unpaired with a zone and re-pairedwith the zone. Such an unpairing may occur if the NMD leaves theenvironment associated with the media playback system (e.g., leaves thehome environment, leaves the zone), moves outside of the range of acommunication path (e.g., communication path 542, 544, and/or 546)(e.g., moves outside of wireless signal range of the communicationpaths), and/or disconnects from the communication path (e.g.,disconnects from a network, logs out of a system or service, disables acommunication interface, enters airplane mode). The NMD may laterre-pair with the zone, for example, if the NMD returns to theenvironment associated with the media playback system (e.g., re-entersthe home environment), returns to within range of a communication path(e.g., moves into wireless signal range of the communication paths),and/or reconnects to the communication path (e.g., establishesconnection to a network, enables a communication interface, exitsairplane mode).

As an example, a user may be carrying the NMD and may remove the NMDfrom the premises of the user's home. Consequentially, the NMD may loseconnection with a zone or device to which the NMD was paired. The NMDmay store this last-known connection or pairing and automaticallyattempt to re-connect or re-pair with this last-known connection orpairing upon returning to the user's home. In some aspects, the NMD maydynamically pair based on location information as described herein, andthe dynamic pairing may occur automatically by the NMD once the NMDdetermines that the NMD is able to pair with a zone or device.

Any voice commands spoken by the user and received by the NMD while theNMD is unpaired may be processed and/or executed immediately or queuedfor processing and/or execution upon re-pairing, and any audio responsesto the voice commands may be played on a dynamically selected device asdescribed herein. For example, the user may play a playlist outside ofthe home and speak a voice command to continue playing the same playlistafter returning home. The NMD may begin the processing of the voicecommand (e.g., begin processing locally, transmit to another device insystem 500 for processing) prior to the user entering the home, prior tore-pairing with any devices or zones in the home, after entering thehome, or after re-pairing with any devices or zones in the home. Theaudio response to the voice command may be played by the first zone ordevice with which the NMD is paired which may be according to dynamicpairing aspects described herein. The same un-pairing and re-pairingfeatures may apply to a user or NMD moving between different zones. Inshifting the pairing between different zones or un-pairing andre-pairing with a particular zone or device, state information ofcurrently playing content or last-played content may be transferred withor separately from the pairing procedure.

In some instances, the voice command may include a particular zone inwhich to execute a command (e.g., play music in the Kitchen). Inresponse to the request in the voice command of a particular zone (e.g.,Kitchen), the NMD can automatically change its pairing to the Kitchenzone. A request for a particular zone in the voice command may overridedynamic pairing selections that the NMD may have otherwise made based onthe detected location of the user or NMD.

Other techniques to determine location information related to a user orother device are described in U.S. patent application Ser. No.13/340,126 filed Dec. 29, 2011 issued as U.S. Pat. No. 9,084,058, U.S.patent application Ser. No. 13/536,250 filed Jun. 28, 2012 issued asU.S. Pat. No. 9,225,307, U.S. patent application Ser. No. 13/568,993filed Aug. 7, 2012 issued as U.S. Pat. No. 8,930,005, U.S. patentapplication Ser. No. 13/777,780 filed Feb. 26, 2013 issued as U.S. Pat.No. 9,195,432, U.S. application Ser. No. 14/475,191 filed Sep. 2, 2014,and U.S. patent application Ser. No. 14/681,735 filed Apr. 8, 2015, thecontents of which are all incorporated by reference in their entireties.

The method may proceed to block 808 to determine whether the selectedzone (e.g., default zone or device(s) or dynamically selected zone ordevice(s)) is grouped with an additional zone as discussed above. If theselected zone is not grouped with another zone, the system 500 can causethe selected zone which can include one or more response devices to playthe audio response at block 810. The audio response may be generated orstored in the device playing the audio response and/or the audioresponse may be transmitted from any other device in system 500 over anyof communication paths 542, 544, 546 to the device playing the audioresponse. The audio response may be received in a message from otherdevices and/or the selected response device(s) may receive a messagefrom a device processing the voice command (e.g., any device in system500) containing an identifier that corresponds to the particular audioresponse to play.

In some aspects, the system may determine that the selected zone (e.g.,predetermined zone, default zone, dynamically selected zone) or theselected device(s) (e.g., predetermined device, default device,dynamically selected device) are in a zone that is grouped with anotherzone and proceed to block 812. For example, the selected zone and theadditional zone grouped with the selected zone may be playing audiocontent synchronously (e.g., playing the same audio contentsimultaneously in sync). While receiving the voice command processingthe voice command, and/or playing the audio response to the voicecommand, playback of currently playing media items or audio content(e.g., songs, playlist, radio station, video) may continue uninterruptedwhile the audio response is played from device(s) of the selected zone.Meanwhile, any media item or audio content which is to be played as aresult of an audio or voice command may be played in the selected zonein addition to any other zones grouped with the selected zone. Forexample, the voice command may be a request to play a particular song,and while the voice command is processed and the song is requested,audio playback of any currently playing audio content may continue whileaudio responses are played by the dynamically selected or predeterminedresponse device(s). Once the voice command is processed and therequested song is identified and retrieved, playback by all playbackdevices of the selected zone and any zones grouped with the default zonemay change to the requested song.

In some aspects, all playback devices in the same zone as the responsedevice and zones grouped with the response device may play the audioresponse. The determination of whether the audio response is to beplayed in these zones may be based on context information (e.g., userpresence). For example, if the selected response device(s) are in afirst zone that is grouped for synchronous audio playback with a secondzone, the system may determine that user(s) are present in the secondzone and play the audio response to notify other user(s) that a receivedvoice command is causing a change in the playback state.

In some aspects, more than one audio response may be provided for asingle voice command. For example, a first audio response may be playedin response to a device of the plurality of devices 500 receiving avoice command to provide acknowledgment or notification to the user thatthe voice command has been received, and a second audio response may beplayed in response to processing and/or completion of processing of thevoice command. The first audio response and second audio response may beplayed on any combination of one or more predetermined devices ordynamically selected devices. For example, a user may speak a voicecommand of playing to play a “Top 40 Playlist,” and a default audioresponse device may play a first audio response (e.g., tone, ringtone,notification sound, sound clip) indicating that the voice command hasbeen received by a NMD (e.g., NMD associated with the default zone). Inresponse to receiving an indication that the voice command is beingprocessed or processing is complete, a second audio response, such as“The Top 40 Playlist is now being played,” may be generated and/orreceived. The second audio response may be played on a dynamicallyselected device. The various blocks of FIGS. 7 and 8 may be repeated, asneeded, for additional audio commands and/or audio responses.

In some aspects, the selected one or more response devices may adjustthe playback volume of the audio response based on various factors. Someof the response device(s) may already be set at a particular volumelevel before receiving the voice command and/or before playing the audioresponse, and the response device(s) may play the audio response at theparticular volume level. The response device(s) may be set at theparticular volume level even though the response device(s) might not becurrently playing any media content. The audio response to the voicecommand may be played at a volume greater than, less than, or equal tothe particular volume level.

In some instances, the system may determine that the selected one ormore zones are still some distance from the user, and the system mayplay the audio response at a louder volume so that the user can hear theaudio response. For example, the system may have selected a playbackdevice in the kitchen zone because the kitchen playback device(s) isclosest to the user, but the user may be located outside in the backyardwhere there is no playback device. In order for the user to hear theaudio response from the playback device, the audio response may need tobe played at a greater volume. The volume of the audio response may beindependent of the volume or loudness at which media content may beplayed. In this instance, the NMD may also be in the kitchen zone, andthe system may determine to use an inverse relationship between theloudness of the user's voice as received by the NMD and volume of theaudio response playback.

The determination of whether to adjust the playback volume may depend oncontext information such as user location and/or user presenceinformation. For example, the system may have determined that the useris greater than a threshold distance from the selected playback deviceor is the user is in a particular zone which may be predetermined to beat a distance where volume of audio response playback is to be adjusted.In some aspects, the system might not increase the volume of the audioresponse playback above a threshold volume level if the systemdetermines that another person or use is in the selected zone.

In some aspects, the system may use other context information such asthe time of day, day of the week, and/or volume of currently playedmedia content when determining the volume at which to play the audioresponse. The other context information may be used to distinguish fromwhether the user is at a distance from the NMD or whether the user isproviding a voice command at a low volume (e.g., whisper). For example,the time of day may be the evening or late at night when other users insame location may be sleeping. The user may not wish to disturb otherusers who are sleeping and may therefore provide a voice command at alow volume. Based on the time of day being after a certain time (e.g., 9pm), the system may determine to switch to a special mode such as anight mode and play the audio response at the same or around the samevolume at which the NMD received the voice command. In some aspects, auser may be provided an option to use the night mode via a controldevice. In yet another example, sleeping times may be more flexible onweekends, and therefore the system may determine that even though thetime of day is after a certain time, the system might not use a nightmode because the day is a weekend.

Another example may be to determine the volume of the audio responsebased in part on the volume at which the currently playing music may beplayed. For example, the system may decide to automatically use a nightmode audio response volume when the volume of music is set below orequal to a threshold volume level. The user may be playing music at alower volume so as to not disturb other people who may be within hearingrange at a higher volume. In this example, there may be a directcorrelation between the volume of the music playback and/or the loudnessof the user's voice and the volume of the audio response.

The system may receive context information from other devices in theenvironment such as presence information and/or location information ofpeople in the environment using various techniques described herein. Thesystem may determine that the volume at which an audio response is to beplayed may be changed from current volume settings based on the contextinformation and/or location information.

The audio response playback volume may be adjusted independent of or incombination with an adjustment of the playback volume of media contentduring the audio response playback period, or the playback volume ofmedia content may be adjusted during the audio response playback periodas described in U.S. application Ser. No. 15/098,760 filed Apr. 14, 2016and entitled “Content Mixing,” the contents of which are incorporated byreference.

In causing the selected one or more playback devices to play an audioresponse with an adjusted volume, a playback device of the one or moreplayback devices and/or one or more computing devices in cloud network502 may determine the adjusted volume. For example, one or more of thecomputing devices in cloud network 502 may determine the adjusted volume(e.g., volume level, gain level, gain adjustment amount) and transmitthe adjust volume information to the selected one or more playbackdevices. In another example, the selected one or more devices maydetermine the adjusted volume.

While the features described herein address selecting one or moredevices to play an audio response to a voice command, the selected oneor more devices may provide a visual indication as a response to thevoice command instead of or in addition to playing the audio response.

IV. Conclusion

The description above discloses, among other things, various examplesystems, methods, apparatus, and articles of manufacture including,among other components, firmware and/or software executed on hardware.It is understood that such examples are merely illustrative and shouldnot be considered as limiting. For example, it is contemplated that anyor all of the firmware, hardware, and/or software aspects or componentscan be embodied exclusively in hardware, exclusively in software,exclusively in firmware, or in any combination of hardware, software,and/or firmware. Accordingly, the examples provided are not the onlyway(s) to implement such systems, methods, apparatus, and/or articles ofmanufacture.

Additionally, references herein to “embodiment” means that a particularfeature, structure, or characteristic described in connection with theembodiment can be included in at least one example embodiment of aninvention. The appearances of this phrase in various places in thespecification are not necessarily all referring to the same embodiment,nor are separate or alternative embodiments mutually exclusive of otherembodiments. As such, the embodiments described herein, explicitly andimplicitly understood by one skilled in the art, can be combined withother embodiments.

The specification is presented largely in terms of illustrativeenvironments, systems, procedures, steps, logic blocks, processing, andother symbolic representations that directly or indirectly resemble theoperations of data processing devices coupled to networks. These processdescriptions and representations are typically used by those skilled inthe art to most effectively convey the substance of their work to othersskilled in the art. Numerous specific details are set forth to provide athorough understanding of the present disclosure. However, it isunderstood to those skilled in the art that certain embodiments of thepresent disclosure can be practiced without certain, specific details.In other instances, well known methods, procedures, components, andcircuitry have not been described in detail to avoid unnecessarilyobscuring aspects of the embodiments. Accordingly, the scope of thepresent disclosure is defined by the appended claims rather than theforgoing description of embodiments.

When any of the appended claims are read to cover a purely softwareand/or firmware implementation, at least one of the elements in at leastone example is hereby expressly defined to include a tangible,non-transitory medium such as a memory, DVD, CD, Blu-ray, and so on,storing the software and/or firmware.

We claim:
 1. A computing device, comprising: at least one processor; anda non-transitory computer-readable medium comprising programinstructions that, when executed by the at least one processor, causethe first playback device to perform functions comprising: receiving viaa network microphone device of a media playback system, a voice commanddetected by at least one microphone of the network microphone device,wherein the media playback system comprises a plurality of zones, andwherein the network microphone device is a member of a default playbackzone; dynamically selecting an audio response zone from the plurality ofzones to play an audio response to the voice input and foregoingselection of the default playback zone, wherein the selected zonecomprises a playback device, and wherein the dynamically selectingcomprises determining that the network microphone device is paired withthe playback device; and causing the playback device of the selectedzone to play the audio response.
 2. The computing device of claim 1,wherein the functions comprise determining that the playback device isgrouped with another playback device in the selected zone, whereincausing the playback device of the selected zone to play the audioresponse comprises causing only the audio response zone to play theaudio response.
 3. The computing device of claim 1, wherein thedynamically selecting further comprises selecting the audio responsezone from the plurality of zones to play the audio response based oncontext information.
 4. The computing device of claim 3, wherein thecontext information comprises a volume level at which the playbackdevice is set to play back media content.
 5. The computing device ofclaim 3, wherein the context information comprises presence informationindicative of a presence of one or more users in an environment of atleast one of the network microphone device and the playback device. 6.The computing device of claim 1, wherein the functions comprise (i)detecting a presence of one or more users and (ii) pairing the networkmicrophone device and the playback device in response to the detecting.7. The computing device of claim 1, wherein the playback device is afirst playback device, the selected zone comprises a second playbackdevice, and the second playback device comprises the network microphonedevice.
 8. A tangible, non-transitory computer-readable medium havingstored thereon instructions that when executed by a processor cause acomputing device to perform functions comprising: receiving via anetwork microphone device of a media playback system, a voice commanddetected by at least one microphone of the network microphone device,wherein the media playback system comprises a plurality of zones, andwherein the network microphone device is a member of a default playbackzone; dynamically selecting an audio response zone from the plurality ofzones to play an audio response to the voice input and foregoingselection of the default playback zone, wherein the selected zonecomprises a playback device, and wherein the dynamically selectingcomprises determining that the network microphone device is paired withthe playback device; and causing the playback device of the selectedzone to play the audio response.
 9. The computer-readable medium ofclaim 8, wherein the functions comprise determining that the playbackdevice is grouped with another playback device in the selected zone,wherein causing the playback device of the selected zone to play theaudio response comprises causing only the audio response zone to playthe audio response.
 10. The computer-readable medium of claim 8, whereinthe dynamically selecting further comprises selecting the audio responsezone from the plurality of zones to play the audio response based oncontext information.
 11. The computer-readable medium of claim 10,wherein the context information comprises a volume level at which theplayback device is set to play back media content.
 12. Thecomputer-readable medium of claim 10, wherein the context informationcomprises presence information indicative of a presence of one or moreusers in an environment of at least one of the network microphone deviceand the playback device.
 13. The computer-readable medium of claim 8,wherein the functions comprise (i) detecting a presence of one or moreusers and (ii) pairing the network microphone device and the playbackdevice in response to the detecting.
 14. The computer-readable medium ofclaim 8, wherein the playback device is a first playback device, theselected zone comprises a second playback device, and the secondplayback device comprises the network microphone device.
 15. A method,comprising: receiving via a network microphone device of a mediaplayback system, a voice command detected by at least one microphone ofthe network microphone device, wherein the media playback systemcomprises a plurality of zones, and wherein the network microphonedevice is a member of a default playback zone; dynamically selecting anaudio response zone from the plurality of zones to play an audioresponse to the voice input and foregoing selection of the defaultplayback zone, wherein the selected zone comprises a playback device,and wherein the dynamically selecting comprises determining that thenetwork microphone device is paired with the playback device; andcausing the playback device of the selected zone to play the audioresponse.
 16. The method of claim 15, further comprising determiningthat the playback device is grouped with another playback device in theselected zone, wherein causing the playback device of the selected zoneto play the audio response comprises causing only the audio responsezone to play the audio response.
 17. The method of claim 15, wherein thedynamically selecting further comprises selecting the audio responsezone from the plurality of zones to play the audio response based oncontext information.
 18. The method of claim 17, wherein the contextinformation comprises a volume level at which the playback device is setto play back media content.
 19. The method of claim 17, wherein thecontext information comprises presence information indicative of apresence of one or more users in an environment of at least one of thenetwork microphone device and the playback device.
 20. The method ofclaim 15, further comprising (i) detecting a presence of one or moreusers and (ii) pairing the network microphone device and the playbackdevice in response to the detecting.